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Abstract  -  Process  Refinement  or  feedback  is  a  key 
component  of  any  closed  loop  system.  Fusion  models 
are  no  exception.  In  this  paper  we  will  explore  what 
process  refinement  means  in  terms  of  the  higher  levels  of 
fusion.  In  doing  so  we  further  refine  the  existing 
definitions  of  the  various  levels  (as  defined  by  the  Joint 
Director  of  Laboratories,  JDL)  and  based  on  these 
definitions  we  discuss  how  each  of  these  levels  interacts 
with  each  other. 
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1  Background 

Process  Refinement,  as  defined  by  the  JDL  panel,  is  an 
ongoing  monitoring  and  assessment  of  the  fusion  process 
to  refine  the  process  itself  and  to  regulate  the  acquisition 
of  data  to  achieve  optimal  results  (Klein,  1993).  Level  4 
(Process  Refinement)  interacts  with  each  of  the  other 
levels.  But  what  does  this  mean?  How  does  it  interact? 
Past  work  has  concentrated  on  Level  0  (Sub-Object  Data 
Assessment)  and  1  (Object  Assessment)  and  their 
interaction  with  Level  4,  but  little  has  been  accomplished 
between  Level  4  and  the  higher  levels  -  2  (Situation 
Assessment)  and  3  (Impact  or  Threat  Assessment).  We 
begin  our  discussion  with  a  number  of  basic  definitions 
from  which  we  will  refine  the  existing  JDL  definitions. 
After  we  present  our  view,  we  next  show  where  and  how 
process  refinement  fits  into  the  overall  framework.  We 
conclude  this  paper  with  a  number  of  issues  and 
challenges. 

1.1  What  is  Level  2/3  Fusion? 

There  continues  to  be  a  debate  as  to  what  Levels  1  and  2 
represent.  One  belief  is  that  Level  1  deals  only  with  the 
tracking  and  identification  of  individual  objects  while 
Level  2  is  the  aggregation  of  the  objects  into  groups  or 
units.  For  example,  Level  1  objects  could  be  various 
equipments  (tanks,  APCs,  missiles,  etc).  At  Level  2, 
equipment  along  with  personnel  can  be  aggregated  into  a 
unit  or  division  based  on  time  and  space.  But  if  we 
consider  this  separation  then  several  questions  arise;  how 
do  we  account  for  concepts  or  non-physical  objects  and 
can’t  we  track  a  group  or  activity  like  an  object?  What  is 
a  situation?  How  does  the  system  acquire  the  necessary  a 


priori  knowledge  (or  relationships)  to  perform 
aggregation?  What  is  the  difference  between  models  for 
identifying  an  object,  a  group  or  an  activity?  To  begin  to 
answer  these  questions  we  first  present  a  number  of  basic 
definitions  and  then  use  them  to  refine  what  we  mean  by 
Level  1  and  2.  We  then  will  explore  the  difference 
between  Level  2/3  and  what  Endsley  [4]  presents  as 
Projection. 

Wikipedia  defines  an  entity  as  “something  that  has  a 
distinct,  separate  existence,  though  it  need  not  be  a 
material  existence.  In  particular,  abstractions  and  legal 
fictions  are  usually  regarded  as  entities.  In  general,  there  is 
also  no  presumption  that  an  entity  is  animate.  The  word 
entity  is  often  useful  when  one  wants  to  refer  to  something 
that  could  be  a  human  being,  a  non-human  animal,  a  non¬ 
thinking  life-form  such  as  a  plant  or  fungus,  a  lifeless 
object,  or  even  a  belief;  for  instance,  one  could  say  that 
any  entity  that  enters  a  black  hole  woidd  be  transported,  in 
many  pieces,  to  another  dimension.”  An  object  is  “a 
physical  entity;  something  that  is  within  the  grasp  of  the 
senses”  (Wikipedia);  “something  perceptible  by  one  or 
more  of  the  senses,  especially  by  vision  or  touch”  (The 
Free  Dictionary).  What  if  the  entity  is  not  a  physical 
object?  How  can  we  describe  it?  Generally  speaking,  an 
abstract  entity  still  can  be  associated  with  a  time  or 
existence  and  an  abstract  concept. 

A  group  is  “a  number  of  things  (entities,  to  include 
individuals)  being  in  some  relation  to  each  other”  while 
an  event  is  “something  that  takes  place;  an  occurrence  at 
an  arbitrary  point  in  time;  something  that  happens  at  a 
given  place  and  time”  (Wikipedia).  Both  entities  and 
groups  can  be  associated  with  a  specific  event  or  events. 
An  activity  is  “something  done  as  an  action  or  a 
movement”  (Wikipedia).  Activities  are  composed  of 
entities/groups  related  by  one  or  more  events  over  time 
and/or  space. 

Thus  by  definition  an  event,  group  and  activity  can  be 
considered  as  a  more  complex  entity  (or  in  terms  of  the 
JDL,  an  object)  and  can  be  tracked  and  identified.  As  a 
side  note,  the  JDL  Lexicon  does  define  an  entity  as  “Any 
object  or  object  set  (or  event  or  event  set)  which  forms  the 
basis  of  a  hypothesis  used  in  data  fusion  processes”  but 
does  not  define  what  an  object  or  event  is.  Now  back  to 
our  discussion,  by  using  the  definitions  presented  above, 
we  argue  that  activities  and  the  aggregation  of  these 
activities  (which  we  refer  to  as  the  situation)  is  both  a  part 
and  a  result  of  Level  1.  Models  or  a  priori  knowledge  is 
necessary  for  level  1  to  be  capable  of  identifying  the 
object,  group  or  activity.  This  a  priori  knowledge  (i.e.,  the 
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relationships  or  associations)  that  can  be  learned  through 
Knowledge  Discovery  and  validated  by  an  operator  or 
provided  directly.  Here  we  note  that  Knowledge 
Discovery  techniques  can  only  learn  statistically  relevant 
occurrences.  As  such,  new  or  novel  ideas  cannot  be 
learned  and  require  knowledge  elicitation. 

Level  2  is  then  the  assessment  of  the  situation  at  that 
snapshot  in  time.  This  includes  the  interpretation  or 
meaning  of  what  is  happening  with  respect  to  context  and 
time  while  Level  3  is  the  detennination  of  whether  there 
exists  a  threat  or  impact  -  is  there  an  entity,  group,  event 
or  activity  that  we  should  care  about?  Specifically, 
situation  assessment  is  a  quantitative  evaluation  of  the 
situation  that  has  to  do  with  the  notions  of  judgment, 
appraisal,  and  relevance.  Roy  [5]  provides  a  description 
of  a  number  of  questions/products  that  are  developed 
under  what  they  call  Situation  Analysis.  In  our  case,  we 
believe  a  number  of  these  products  are  created  at  Level  1 
while  others  are  Level  2.  Level  1  attempts  to  answer  such 
questions  as  Existence  and  Size  Analysis  (How  Many?), 


Identity  Analysis  (What/Who?),  Kinematics  Analysis 
(Where?),  and  When,  while  Level  2  provides:  Behavior 
Analysis  (What  is  the  object  doing?),  Activity  Level 
Analysis  (Build  up,  draw  down?),  Intent  Analysis  (Why?), 
Salience  Analysis  (What  makes  it  important?),  and 
Capability/Capacity  Analysis  (What  could  they/it  do?). 
We  can  also  argue  that  Level  2  and  3  are  a  result  of 
analysis  of  current  data.  After  this  assessment,  the  next 
step  would  be  to  forecast  or  project  the  current  situation 
and  threat  into  the  future.  We  specifically  call  this 
function  as  projection  (as  defined  by  Endsley  [4])  and 
show  it  as  a  separate  capability  from  Level  2/3  of  the  JDL. 
It  is  true,  however  that  to  perform  projection  one  would 
rely  on  the  current  situation  and  forecast  not  only  those 
salient  activities  into  the  future  but  also  any  future  impacts 
or  threats.  Thus,  to  summarize,  a  situation  is  a  snapshot  of 
the  aggregated  activities  at  time  t.  Projection  takes  the 
situation  and  projects  or  forecasts  it  to  time  t  +  n,  where  n 
is  some  number  of  time  steps.  Figure  1  summarizes  what 
we  have  presented  thus  far. 
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FIGURE  1  -  Proposed  Refinement  of  JDL  Model 


2  Process  Refinement 

In  this  next  section  we  discuss  what  Process  Refinement 
means  for  Level  2/3  and  conceptually  how  it  can  be 
implemented.  We  also  present  how  it  is  affected  by 
Projection.  The  basic  definition  of  Process  Refinement  as 
presented  above  covers  two  separate  but  integrated 
capabilities.  For  the  purpose  of  our  discussion  we  will 
divide  them  into  an  external  and  internal  process. 
Externally,  we  are  concerned  with  providing  sensors  or 
collections  with  positioning  information  based  on 
forecasted  or  anticipated  movement  of  objects/entities  or 
groups.  The  classical  example  here  is  the  tracking  of  an 
object.  A  common  tracking  algorithm  used  in  today’s 
system  is  a  Kalman  Filter.  Kalman  Filters  provide  the 
ability  to  forecast  where  the  object  could  be  in  one  time 
increment  in  the  future.  This  position  information  can 
then  be  provided  to  “better”  position  the  sensor. 
Theoretically,  a  similar  approach  can  be  done  with 
concepts  and  groups.  Also,  since  models  are  typically 
used  to  define  relationships,  one  can  also  use  these  models 
for  projection.  Models  that  describe  activities  not  only 


describe  how  entities,  groups  and  events  are  related,  they 
also  provide  knowledge  as  to  what  might  happen  next  and 
thus  can  provide  positional  information  for  sensor 
collection.  We  define  models  in  a  generic  sense  (a  body 
of  knowledge  or  relationships,  technique  independent). 
They  can  be  kinematics  models  describing  how  a 
particular  piece  of  equipment  behaves,  what  an  object 
looks  like  for  identification  (as  in  automatic  target 
recognition),  one  that  identifies  a  specific  social  group 
(terrorist  cell  or  a  specific  military  unit  or  division)  or  a 
number  of  inter-related  events  that  make  up  an  activity. 

Now  back  to  our  discussion.  Level  2/3  is  concerned 
with  identification  of  the  current  situation,  identification 
of  the  adversary’s  capabilities  and  analysis  of  the 
adversary’s  strengths  and  weaknesses.  This  understanding 
of  the  adversary  is  packaged  into  what  is  called  an 
adversarial  model.  In  today’s  environment,  models  as 
defined  by  current  doctrine  consist  of:  (1)  Doctrinal 
Templates  (illustrate  the  employment  patterns/dispositions 
preferred  by  an  adversary  when  not  constrained  by  the 
effects  of  the  operational  environment);  (2)  Description  of 
Adversary  Tactics  and  Options  (a  written  description  of  an 


opponent’s  preferred  tactics)  and  (3)  Identification  of 
High- Valued  Targets  (those  assets  that  the  adversary 
commander  requires  for  the  successful  completion  of  their 
mission). 

Projection,  Anticipation  or  Forecasting  is  accomplished 
by  the  analyst  and  supports  the  development  or  analysis  of 
possible  (1)  adversary  intent;  (2)  Courses  of  Action 
(COA)  -  to  include  a  prioritize  list  identifying  the  most 
likely  and  most  dangerous  and  (3)  a  set  of  collection 
requirements.  As  part  of  the  process  that  an  analyst 
performs  while  developing  their  situation  and 
impact/threat  assessment,  they  may  develop  a  collection 
of  requirements  and  identify  new  relationships  (and  in 
turn  update  their  model(s)  of  the  world).  As  they  are 
developing  possible  adversarial  intents  and  COAs  the 
analyst  determines  the  key  events  in  each  to  aid  in  the 
determination  as  to  which  one  of  the  proposed  COAs  is 
unfolding  (or  not).  These  key  events  are  then  turned  into 
a  set  of  collection  requirements  and  used  as  part  of  the 
process  refinement  process 

The  process  described  above  assumes  that  we  have 
perfect  collection  and  retrieval  technology  and  that  we 
obtain  only  the  data/information  that  we  need  to  perform 
our  job.  No  manner  the  technique,  it  is  prone  to  errors  and 
depending  on  how  requests  are  written  can  either  open  a 
huge  flood  gate  or  miss  that  one  important  piece  of  data. 
One  must  always  provide  the  ability  for  the  analyst  to 
examine  existing  archives  retrospectively  to  see  if 
additional  evidence  is  available  about  a  certain  object, 
group,  event  or  activity.  Existing  data  found  by  sifting 
through  the  raw  data  and  not  automatically  delivered  by 
the  existing  profiles  could  lead  to  the  profiles  being 
revised  or  new  ones  being  created. 

Internal  processes  also  need  to  be  monitored  to  ensure  that 
the  information  processing  system  is  performing  as 
designed.  At  the  object  level  one  can  suggest,  possibly 
based  on  environmental  inputs,  which  source  is  “better”  at 
that  time  for  tracking  or  identifying  the  object  or  sending 
the  same  sensor  data  to  multiple  algorithms  (running  in 
parallel),  coming  up  with  possibly  different  answers  and 
combining  the  results  in  some  manner.  Similar  concepts 
can  be  used  at  the  activity  level.  As  previously  mentioned 
a  second  area  is  the  update  of  a  priori  knowledge  or 
models.  As  new  information  comes  in  and  new 
knowledge  is  developed  through  the  assessment  and 
projection  process,  the  analyst  may  update  existing 
models  or  add  create  new  models  (regardless  of  whether  it 
is  a  new/modified  object,  group  or  activity). 

3  Conclusion 

The  purpose  of  this  paper  was  to  present  a  view,  based  on 
a  number  of  years  of  building  systems  that  describes  a 
more  logical  functional  boundary  for  the  JDL  fusion 
model.  The  model  has  served  this  community  well  but  as 
time  and  understanding  has  progressed,  so  must  the 
model.  We  began  our  discussion  with  a  number  of  basic 
definitions  followed  by  a  description  of  various  types  of 
interaction  between  Levels  4  and  2/3.  We  refined  the 
existing  definitions  of  Level  1  and  2  in  hopes  to  better 
clarify  each  of  their  roles.  We  also  provided  a  discussion 
describing  a  distinction  between  Level  2/3  as  described  by 


the  JDL  (for  the  current  situation)  and  Endsley’s  concept 

of  projection  and  how  they  can  jointly  exist. 
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